<html>
<head><meta charset="utf-8"><title>Moving the `odht` crate to rust-lang? · t-compiler/wg-incr-comp · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/index.html">t-compiler/wg-incr-comp</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Moving.20the.20.60odht.60.20crate.20to.20rust-lang.3F.html">Moving the `odht` crate to rust-lang?</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="231005244"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Moving%20the%20%60odht%60%20crate%20to%20rust-lang%3F/near/231005244" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Moving.20the.20.60odht.60.20crate.20to.20rust-lang.3F.html#231005244">(Mar 19 2021 at 10:57)</a>:</h4>
<p>Dear <span class="user-group-mention" data-user-group-id="3282">@wg-incr-comp</span>,</p>
<p>As some of you might have seen I've been working on simplifying and optimizing how incr. comp. handles <code>DefPathHash</code> to <code>DefId</code> mapping in <a href="https://github.com/rust-lang/rust/pull/82183">https://github.com/rust-lang/rust/pull/82183</a>. Both the reduction of complexity and the performance improvements look very promising -- so I'd like to move forward with getting the PR actually merged.</p>
<p>The main reason why the PR can simplify the implementation while at the same time improve performance is that it relies on a hash table implementation that does not need any upfront decoding. This is a very useful thing to have. With memory mapped files the compiler will even only load those parts of the hash table from the disk that are actually accessed. I imagine that we can find quite a few more use cases for a data structure like this.</p>
<p>The on-disk hash table is implemented in <a href="https://github.com/michaelwoerister/odht">https://github.com/michaelwoerister/odht</a> and, in accordance to <a href="https://rust-lang.github.io/compiler-team/procedures/crates/">https://rust-lang.github.io/compiler-team/procedures/crates/</a>, I would like to propose moving this crate into the rust-lang org. I think the crate can be generally useful throughout the compiler and won't need a lot of maintenance once everything is set up.</p>
<p>What do you think?</p>



<a name="238032151"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Moving%20the%20%60odht%60%20crate%20to%20rust-lang%3F/near/238032151" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cjgillot <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Moving.20the.20.60odht.60.20crate.20to.20rust-lang.3F.html#238032151">(May 09 2021 at 12:17)</a>:</h4>
<p>I think <a href="https://github.com/rust-lang/rust/issues/82183">#82183</a> is a great move forward, and that odht may eventually help with the hygiene interning table also. What is the required review process to move odht into rust-lang? I have been looking into the implementation, but I do not have a comprehensive understanding yet.</p>



<a name="244968991"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Moving%20the%20%60odht%60%20crate%20to%20rust-lang%3F/near/244968991" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cjgillot <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Moving.20the.20.60odht.60.20crate.20to.20rust-lang.3F.html#244968991">(Jul 05 2021 at 19:52)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span> What do you need to move this forward? Is a formal review process necessary?</p>



<a name="245014071"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Moving%20the%20%60odht%60%20crate%20to%20rust-lang%3F/near/245014071" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Moving.20the.20.60odht.60.20crate.20to.20rust-lang.3F.html#245014071">(Jul 06 2021 at 09:22)</a>:</h4>
<p>The <a href="https://github.com/rust-lang/compiler-team/issues/438">MCP</a> has been accepted so we just need to transfer the crate to the rust-lang org and do a <a href="http://crates.io">crates.io</a> release. It would indeed be helpful if someone could do a review of the crate. The fact that it had another pair of eyes on it could be documented in one of the first PRs that makes use of it, I'd say. Or even better, we add at least one co-owner to the Cargo.toml after the transfer. </p>
<p><span class="user-mention" data-user-id="248906">@cjgillot</span>, if you want to take a look at the code that would be very helpful indeed. It tried to add tests and documentation but it is certainly a good idea to read up on how SwissTable works (e.g. here: <a href="https://gankra.github.io/blah/hashbrown-tldr/">https://gankra.github.io/blah/hashbrown-tldr/</a>). That will make it a lot easier to follow the code, I'm sure.</p>
<p><span class="user-mention" data-user-id="125250">@Wesley Wiser</span>, do you agree that that would be sufficient in terms of giving the crate an initial review?</p>



<a name="245040464"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Moving%20the%20%60odht%60%20crate%20to%20rust-lang%3F/near/245040464" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Moving.20the.20.60odht.60.20crate.20to.20rust-lang.3F.html#245040464">(Jul 06 2021 at 13:46)</a>:</h4>
<p>I think that would be fine. I can also take a look if you don't have time <span class="user-mention" data-user-id="248906">@cjgillot</span></p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>